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Abstract. In this note, it is shown that if /: F^' — > is any 
function and A = (Ai, . . . , An) is uniformly distributed over F^, 
then the average over (fci, . . . , fc„) e F^' of the Rcnyi (and hence, of 
the Shannon) entropy of f{A) + {kiAi, . . . , fc„yl„) is at least about 
log2((7") — n bits. In fact, it is shown that the average collision 
probability of /(A) + {kiAi, . . . , fc„^„) is at most about 



1. Introduction 

Suppose that / : Fg — )■ Fg is an arbitrary function (where g is a prime 
power and Fg is the finite field of q elements). Let A be a random 
variable uniformly distributed over Fg. Clearly, f{A) may be far from 
uniform, while kA is uniform for all A; G F*. Is f{A) + kA nearly 
uniform for most values oi k G F^? More generally, given a positive 
integer n, for an arbitrary / : F^ — )■ F^ and for A uniformly distributed 
over F^, i^ f{A) + {kiAi, . . . , knAn) nearly uniform for most values of 
k e F^? 

Recall that the Shannon entropy H{B) of a random variable B taking 
values in a finite set 5* is defined H{B) := — X]s Pr(_B=s)^o Pr(i? = 
s) ■ log(Pr(i? = s)), while the collision probability of B, cp{B), is de- 
fined by cp{B) := ^^g^Pr(i? = s)^ = Pr(i? = B'), where B' is an 
independent copy of B. The Renyi entropy of B, H2{B), is defined 
by H2{B) := — log(cp(5)). A straightforward application of Jensen's 
inequality shows that H2{B) < H{B). 

Since both the Renyi entropy and the Shannon entropy measure 
randomness (where for both entropies the maximum possible value of 
logdS*!) is equivalent to having uniform distribution, and the minimum 
possible value of is equivalent to being deterministic), a possible for- 
mal phrasing of the above question on f{A) + {kiAi, . . . , knAn) is: How 
much smaller than log(g") might the average over k of the Renyi (or 
Shannon) entropy be? 



""^Throughout, we write Xi for the ith coordinate of a vector x. Also, for a function 
/ with codomain F^', we will write fi for the ith component of / (post-composition 
of / with the ith projection) 

^From this point on, all logarithms are to the base of 2. 
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The collision probability itself is yet an additional measure of ran- 
domness, where the minimum collision probability of 1/\S\ is equiva- 
lent to having uniform distribution and the maximum possible collision 
probability of 1 is equivalent to being deterministic. So, another pos- 
sible formal phrasing of the question on /(A) + {kiAi, . . . , knAn) is: 
How much larger than might the average over k of the collision 
probability be? 

The main motivation for this question is a certain side-information 
problem in information theory [8]. Several neighboring questions were 
considered in the literature. For example, the case n = 1 of Theorem [T] 
ahead extends Lemma 21 of stating that for any /: — t- there 
exists k G ¥q for which \{f{x) + kx\x G F^}! > q/2u The same case 
of Theorem [T] ahead also extends the main theorem of pj, which states 
that the average over G Fg of \{f{x) + kx\x G Fg}| (for / a polynomial 
of degree < char(Fg)) is at least g/(2 — 1/q). In addition, a somewhat 
similar question, concerning the min-entropy of ai ■ f{A) + a2 ■ A for 
random ai and aoin ¥q and for large q was implicitly considered in the 
merger literaturejj see, e.g.. Sec. 3.1 of [3], and Theorem 18 of [3]. 

The main contributions of the current note are the following two 
theorems. 

Theorem 1. Let n > 1 be an integer, let /: F^ — )■ F^ be an arbitrary 
function, and for k G F^ , let : F^ — F" be defined by 

gk{x) := fix) + {kiXi, k2X2, . . . , k^Xn). 

Suppose that a random variable A is uniformly distributed over F^. 
Then 

\j2H^(9k{A)) > log(g")-nlog(2--y 

The point of the theorem is that the average over k of H2{gk{A)) is 
at most about n bits below the entropy of a uniform distribution over 
Fg, regardless of q and /. Of course, since the Shannon entropy is not 
smaller than the Renyi entropy, we may replace H2 by H in Theorem 
[TJ In fact, a stronger result is proven: 



It should be noted that in this case (n = 1), the result follows immediately from 
the Leftover Hash Lemma as described, e.g., in Lemma 7.1 of 7 , or in Theorem 8 

oim- 

^The distribution of ai and 02 depends on whether the merger in question is the 
linear merger or the curve merger, see, e.g., the introduction of [S]. For example, 
for the curve merger of [S], it was shown in [3] that for any e,5 > 0, the weighted 
sum is e-close (in statistical distance) to having min-entropy {1 ~ S) ■ n ■ \og{q), as 
long as (7 > (A/e)'^/^. 
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Theorem 2. Using the terminology of TheoremUl we have 

1 1 / 1 \ " 

(1) J2<:MA))<-l2--] , 



with equality if for all i, fi{x) depends only on Xj. 
Note that by Jensen, 

1 5^ = -5^1log(cp(^,(A))) 



^ feSF" k ^ 



> 



and hence Theorem [2] implies Theorem [H 

As stated in the theorem itself, the bound of Theorem [2]is tight. The 
bound of Theorem [T] is also tight, as seen by the following proposition. 

Proposition 3. For the function /: — )• defined by f{x) := 
{xl, . . . , xl), we have (using the terminology of TheoremU\) 

^ J2 H{gu{A)) = log(g" 

and 



qii i — / \ q 



1 , , f log(g") (1 - J) if q IS even, 

9'" ^^^n I log(g") - n log f 2 - M otherwise. 

2. Proof of Theorem [2] 

The proof begins as the proof of the Leftover Hash Lemma as appear- 
ing in [2] . Letting K and A' be random variables uniformly distributed 
over Fg such that A, K and A' are jointly independent, the left-hand 
side of ([1]) can be written as 

1- J2 cpig^iA)) = 5^ Pr(K = fc) ■ Pr (<7fc(A) = <7fc(A')) 

= 5^Pr(K = fc)-Pr((7x(A) = (?K(A')|K = fc) 

fceFj 

= FT{gK{A)=gK{A')). 

It follows that Theorem |2] is an immediate consequence of the fol- 
lowing Lemma. 

Lemma 4. Using the above notation, 

Fr{g^{A)=9^{A'))<^i^2-^J 
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with equality if for all i, fi{x) depends only on x,. 

Proof. For x, x' G F^, let d^ix, x') be the Hamming distance between 
X and x' (number of coordinates i for which Xi 7^ x-) and let xQx' := 
{xix[, . . . , Xnx'j^). We have 



Pt {gK{A) = gK{A')) = Pt{A = A') + Y,P<MA, A') = d) ■ 



d=i 



(2) 



Pr {gK{A) = gK{A')\du{A, A') = d). 



Now, 



Pr {gK{A) = gK{A')\dn{A,A') = d) 

(probability over K, A and A') is the average over pairs of vectors 
a, a' G F^ of Hamming distance d of expressions like 

(3) Pi {f{a) + K Qa = f {a') + KQa') 

(probability over K). The last expression is either (if fi{a) 7^ fi{a,') 
for some i for which Oj = or q^-^ jq^ otherwise {d entries of K are 
determined by the equation, and the other n — d entries are free). So, 
in either case, the expression in ([3]) is < q~'^ (with equality if for all 
fi depends only on the zth argument), and hence so is the average of 
these expressions. Substituting in ([2]), we get 



Pt {gK{A) = gK{A')) < cp(A) + ^ 



d=l 



— + 



1 1 

— + — 

qn qn 



1 " 

-E 



d=l 



- 2-- 

n \ q 



with equality if for all i, fi{x) depends only on Xi 



□ 



3. Proof of proposition [3] 

The assertion regarding the average Shannon entropy will follow im- 
mediately from the chain rule for conditional Shannon entropy if we 



^Note that this cannot happen if for ah i, fi{x) depends only on Xi. This will 
show that for such functions we have equality in the proposition. 
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prove that for n = 1 and for the function /: — )■ defined by 
f{x) = x^, we have 

(4) -J2H{9k{A))=\og{q)-(l~-) 

for A uniformly distributed on Fg. 

Suppose first that q is even. Then (70 = (s i— )■ x^) is a permutation 
on ¥q (in fact, an automorphism), and so H^qq^A)) = log(g). For 
k,y G ¥g, let X^^y := g^^{y). We claim that for all A; G F* and for all 
7/ e Fq with Xk^y 7^ 0, there are exactly 2 elements in Xk,y'- On one 
hand, there are at most two solutions to a quadratic equation, and on 
the other hand, for x G X^^y, x + is different from x and satisfies 
gk{x + k) = gk{x), which means that x + A; G X^^y. Hence in the case of 
characteristic 2, the average entropy is (l/g)-log(g) + (l — l/g)-log(g/2), 
as desired. 

For odd q, we claim that for all A; G Fg, there is a single y with 
\Xk,y\ = 1, and (g — l)/2 values of y with \Xk^y\ = 2: Fix k, take y 
with Xk^y 7^ 0, and let x G Xk^y. Clearly, gk{—k — x) = gk{x), and 
if X 7^ —k/2, then —k — x 7^ x, which implies that \Xk^y\ = 2. For y 
with — fc/2 G Xk^y, \Xk^y\ must therefore be odd, and hence necessarily 
equal^ 1. Hence in the case of odd characteristic, the average entropy 
is {{q - l)/2) • (2/g) ■ log(g/2) + (1/g) • log(g), as in ©. 

It remains to calculate the average Renyi entropy for / = (a; 1— )■ 
(xf , . . . , x^)). It follows from the above discussion on the Shannon 
entropy that if q is even, then for all k and all i, the collision probability 
of the i-th entry of equals 2/g if fcj 7^ (uniform distribution on 

q/2 elements), and 1/g if fcj = 0. As the collision probability of a vector 
of jointly independent random variables is the product of the individual 
collision probabilities, it follows that cp{gk{A)) = 2'"^'''> / q"' , where w(k) 
is the Hamming weight of k (number of nonzero coordinates in k). 

Sinc^ J2ke¥^ ^i^) = "'0'" ~ nq"^'^, we get 

l5^i7.(,.(A)) = ^^Y.(^og{qn-w{k)) 
^ k ^ fc 

= \og{q-) - l^{nq- - nq--') 
= log(g") - n fl - - 



as desired. 



Of course, the last y equals — and the fact that |^fc.j,| = 1 for this y may 
also be verified directly. 

'''One way to verify the following identity is to note that the sum Wq{n) of the 
weights of all vectors in satisfies Wq(l) ~ q — 1 and Wq{n) = Wq{n — 1) + (g — 
1) • {Wq{n - 1) + q"-i) for n>2. 
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Finally, if q is odd, then it follows from the discussion in the begin- 
ning of the proof that for all fc, the collision probability of any entry 
of 5'fc(A) equals 

1 ^ g-lpy _ 2g-l 
2 \q ) q^ 

Because the collision probability of Qki-^) is the product of the collision 
probabilities of the individual entries, it follows that for all A;, 

HMA)) = - log (i^ . (2g - 1)") = - log (1 . (2 - , 

which completes the proof. 

Remark. Note that in Proposition |3l the components fi may be any 
quadratic functions Xi ^ aixf+biXi+Ci with Oi 7^ for all i (eliminating 
and Ci is done by an invertible function, and then the linear term is 
"absorbed" in the averaging over ki). 
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